feat: Add clinical-note-red-flag-scanner template - #296
Conversation
Adds a clinical documentation compliance analysis template that accepts a clinical note as text and returns a structured, severity-ranked JSON list of documentation red flags (missing consent, drug interactions, incomplete vitals, ambiguous dosing, etc.) for clinician and compliance reviewer triage. - Single-flow template (API Request -> LLM Analyse Note -> Variables -> API Response) - 10 red-flag categories with 4 severity levels (CRITICAL/HIGH/MEDIUM/LOW) - Uses Gemini (gemini-3.1-flash-lite-preview) for analysis - Structured JSON output with per-flag reasoning, location, and remediation - Includes comprehensive agent.md, README with tested example I/O, and safety constitution
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughChangesAdds a Clinical Note Red-Flag Scanner template with Gemini analysis, structured JSON output rules, a four-node execution flow, safety guardrails, documentation, model configuration, and Lamatic packaging metadata. Clinical Note Red-Flag Scanner
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
:robot_face: AgentKit Structural ValidationNew Contributions Detected
Check Results
🎉 All checks passed! This contribution follows the AgentKit structure. |
There was a problem hiding this comment.
Actionable comments posted: 13
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/clinical-note-red-flag-scanner/agent.md`:
- Around line 15-19: Revise the standards-alignment section and guardrail claims
in kits/clinical-note-red-flag-scanner/agent.md at lines 15-19 to state only
that the template may support documentation-review workflows, and explicitly
disclaim regulatory compliance certification. Apply the same narrowed wording
and disclaimer to the user-facing compliance statement in
kits/clinical-note-red-flag-scanner/README.md at lines 12-14.
- Around line 29-33: Update the input-note guidance in the agent documentation
to remove the “any length” claim. Define and enforce a maximum supported note
size or implement chunking before scanning, document the actual limit and
behavior for oversized notes, and keep the required clinicalNote input contract
unchanged.
In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts`:
- Around line 159-163: Update the finalText mapping in the Variables node to
reference only LLMNode_453.output.generatedResponse, removing the trailing
literal “{” so the complete model response remains valid JSON.
- Around line 110-115: Update the API Request trigger configuration for nodeName
"API Request" so advance_schema requires clinicalNote to be a non-empty,
non-whitespace string. If the schema cannot express trimming, add an explicit
guard immediately before the LLM node that rejects empty or whitespace-only
clinicalNote values before scanning.
In `@kits/clinical-note-red-flag-scanner/lamatic.config.ts`:
- Around line 9-12: Add the deployment environment key to the mandatory step
configuration identified by id "clinical-note-red-flag-scanner", using the step
ID expected by runtime flow resolution so it can read the deployed flow ID from
process.env. Preserve the existing id and type values.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md`:
- Line 13: Update the REGULATORY guidance in the clinical-note red-flag scanner
prompt to require jurisdiction and care-setting context before emitting
regulatory findings. If those inputs are unavailable, restrict findings to
requirements explicitly identified by the caller or documented in the note; do
not make jurisdiction-specific claims from clinicalNote alone.
- Line 5: Update the DRUG_INTERACTION category definition in the system prompt
to avoid claiming definitive or exhaustive detection without an authoritative
interaction source; either specify a versioned, clinically verified interaction
database/tool, or restrict findings to interactions explicitly documented in the
note and label screening as non-exhaustive.
- Around line 23-27: Update the recommendation field instructions and the
prohibition near the medical-advice guidance to require documentation-only
remediation actions. Explicitly prohibit treatment choices, dosing regimens,
diagnoses, and triage or emergency-care advice, and revise the README example to
remove nitroglycerin and emergency-care instructions.
- Around line 15-30: Update the JSON response contract in the system prompt to
explicitly require that flagCount equals flags.length, flag IDs are unique and
sequential starting at FLAG-001, and category and severity values use only the
defined valid codes. Require responses to satisfy these invariants in addition
to the existing structure.
- Around line 37-42: Update the evidence guidance near the flag-generation rules
so `detail` and `location` use redacted or paraphrased evidence rather than
reproducing exact patient text. Explicitly prohibit names, MRNs, DOBs, contact
information, and other identifiers while preserving enough non-identifying
context to support each flag.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md`:
- Around line 1-2: Update the clinicalNote interpolation in the clinical-note
scanner prompt to prevent raw PHI from being sent to Gemini. Require a
pre-redacted or de-identified note, or enforce a documented compliant
zero-retention provider configuration before forwarding the content; do not rely
on prompt-level confidentiality instructions.
- Around line 1-2: Update the clinical-note prompt content around
triggerNode_1.output.clinicalNote to clearly delimit the note as untrusted data
and explicitly instruct the model to ignore any commands or instructions
contained within it, while preserving the required JSON, confidentiality, and
safety behavior.
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Around line 1-8: Move the “# Clinical Note Red-Flag Scanner” H1 above the
deploy badge HTML so it is the README’s first content and satisfies MD041. Keep
the existing badge markup unchanged below the heading.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 83786c0b-8632-4fb4-8f70-44be325cb8c6
📒 Files selected for processing (9)
kits/clinical-note-red-flag-scanner/.gitignorekits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/constitutions/default.mdkits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.tskits/clinical-note-red-flag-scanner/lamatic.config.tskits/clinical-note-red-flag-scanner/model-configs/clinical-note-red-flag-scanner_llmnode-453_generative-model-name.tskits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md
- Fix trailing brace in variablesNode mapping - Add envKey to lamatic.config.ts - Add minLength validation to flow advance_schema - Narrow compliance claims and add data privacy warnings in README and agent.md - Add token limit to agent.md - Restrict system prompt DRUG_INTERACTION and REGULATORY claims - Enforce documentation-only recommendations and PII redaction - Add prompt injection defense to user prompt - Enforce strict JSON invariants - Fix markdownlint warnings
There was a problem hiding this comment.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
kits/clinical-note-red-flag-scanner/README.md (3)
67-69: 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick winKeep the public example fully synthetic and non-identifying.
The sample uses patient-like clinical data and demonstrates reproducing MRN/DOB fields, contradicting the redaction requirements. Replace it with clearly synthetic, de-identified content and avoid showing identifier reproduction.
Also applies to: 87-89
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 67 - 69, Replace the clinicalNote example in the README with clearly synthetic, non-identifying content that does not resemble real patient data or include names, demographics, dates of birth, MRNs, or other identifiers. Update the related example at the referenced reproduction section as well, removing any demonstration of identifier reproduction while preserving the red-flag scanner usage context.
105-116: 🔒 Security & Privacy | 🟠 MajorRemove treatment and triage advice from the example output.
The sample recommends medication dosing and emergency-care instructions, while the system contract permits documentation-only remediation. Revise the example to use actions such as documenting missing fields, not treatment or return-to-ED guidance.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 105 - 116, Update the example output entries FLAG-003 and FLAG-004 in the README so their recommendation text only instructs users to document missing clinical fields or plan details. Remove medication dosing guidance and emergency-care or return-to-ED instructions while preserving the existing finding context.
73-80: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winAlign the documented response shape with the runtime contract.
This section says
resultcontains a JSON object and shows the report without the outerresultfield, while the flow and agent documentation defineresultas a JSON string. Show the actual wrapper and serialization, or change the implementation and all documentation consistently.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 73 - 80, Update the “Example Output” section to match the runtime contract: show the outer result field and represent its value as the serialized JSON string returned by the flow. Keep the documented summary and flagCount payload consistent with the implementation and related flow/agent documentation.
♻️ Duplicate comments (1)
kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts (1)
114-114: 🩺 Stability & Availability | 🟠 MajorEnforce the documented maximum note size.
The runtime accepts arbitrarily large notes while the documentation promises a 10,000-token limit. Add bounded validation or chunking before the single LLM request.
kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts#L114-L114: add a maximum-size guard or chunking path to the trigger contract.kits/clinical-note-red-flag-scanner/agent.md#L33-L33: document the enforced limit and oversized-input behavior.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts` at line 114, Enforce the documented 10,000-token maximum in the clinical note trigger contract around the advance_schema definition by adding bounded validation or chunking before the single LLM request. In kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts:114, reject or appropriately process oversized clinicalNote values; in kits/clinical-note-red-flag-scanner/agent.md:33, document the enforced limit and the behavior for oversized input.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md`:
- Line 13: Update the REGULATORY guidance in the system prompt to require
jurisdiction and care setting before making jurisdiction-specific findings. When
those inputs are unavailable, limit findings to requirements explicitly
identified by the caller or clinical note and label them non-jurisdictional.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md`:
- Line 4: Update the prompt input around clinicalNote so raw clinical notes
cannot be forwarded directly to Gemini without a data-control boundary. Require
and validate pre-redacted/de-identified content, or verify a compliant
zero-retention provider configuration before executing the node; otherwise block
execution rather than interpolating the complete note.
---
Outside diff comments:
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Around line 67-69: Replace the clinicalNote example in the README with clearly
synthetic, non-identifying content that does not resemble real patient data or
include names, demographics, dates of birth, MRNs, or other identifiers. Update
the related example at the referenced reproduction section as well, removing any
demonstration of identifier reproduction while preserving the red-flag scanner
usage context.
- Around line 105-116: Update the example output entries FLAG-003 and FLAG-004
in the README so their recommendation text only instructs users to document
missing clinical fields or plan details. Remove medication dosing guidance and
emergency-care or return-to-ED instructions while preserving the existing
finding context.
- Around line 73-80: Update the “Example Output” section to match the runtime
contract: show the outer result field and represent its value as the serialized
JSON string returned by the flow. Keep the documented summary and flagCount
payload consistent with the implementation and related flow/agent documentation.
---
Duplicate comments:
In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts`:
- Line 114: Enforce the documented 10,000-token maximum in the clinical note
trigger contract around the advance_schema definition by adding bounded
validation or chunking before the single LLM request. In
kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts:114,
reject or appropriately process oversized clinicalNote values; in
kits/clinical-note-red-flag-scanner/agent.md:33, document the enforced limit and
the behavior for oversized input.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: fe830c49-c5c0-4bba-bc7e-97d970a0f9b7
📒 Files selected for processing (6)
kits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.tskits/clinical-note-red-flag-scanner/lamatic.config.tskits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md
- Replace realistic PHI with synthetic placeholders in README example - Strip remaining clinical advice from FLAG-003 and FLAG-004 examples in README - Wrap JSON example output in stringified result field in README - Enforce 45000 character limit in flow advance_schema - Document enforced token limit and rejection behavior in agent.md - Strengthen REGULATORY caveat in system prompt (advisory only, caller provides jurisdiction) - Add hard PHI input contract to user prompt
|
/validate |
|
📡 Running Studio validation — results will appear here shortly. |
Studio Runtime Validation (Phase 2)✅ Studio validation passed. The kit loaded successfully in Lamatic Studio. This PR is ready for final review and merge. |
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
kits/clinical-note-red-flag-scanner/README.md (2)
20-20:⚠️ Potential issue | 🟠 MajorMission-critical: align interaction detection with the non-exhaustive model contract.
The README claims “known unaddressed interactions,” while the system prompt correctly states that this is non-exhaustive screening without a verified interaction database. Reword this as interactions explicitly documented or suggested by the note, and state that it is not definitive interaction checking.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` at line 20, Update the DRUG_INTERACTION description in the README to describe only medication interactions explicitly documented or suggested by the clinical note, and clearly state that the scanner does not provide definitive or exhaustive interaction checking.
75-80: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winDescribe
resultas a JSON string, not a JSON object.The example correctly wraps the serialized report inside
"result", but the prose currently says it contains a JSON object. Document it as a string containing JSON to match the flow andagent.md.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 75 - 80, Update the README prose describing the flow’s result field to state that result is a JSON string containing the serialized report, not a JSON object. Keep the existing example and JSON structure unchanged.kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts (1)
161-181: 🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy liftValidate the LLM report before returning it.
The model’s raw
generatedResponseis copied directly into the API result. If Gemini emits markdown, malformed JSON, or violatesflagCount/ID/category invariants, the documented response contract is broken. Add schema-backed structured output or parse and reject invalid reports beforeAPI Response.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts` around lines 161 - 181, Validate and normalize the LLM report before the responseNode named “API Response” returns it: parse the generatedResponse from LLMNode_453, enforce the documented schema and flagCount/ID/category invariants, and reject malformed or markdown-wrapped output. Update the variablesNode_197 mapping so outputMapping uses only the validated report, preserving the existing finalText/result contract for valid responses.
♻️ Duplicate comments (2)
kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts (2)
114-114:⚠️ Potential issue | 🟠 MajorProvide jurisdiction context before emitting regulatory findings.
The flow accepts only
clinicalNote, but the prompt requires caller-supplied jurisdiction and care-setting context. Add those inputs, or restrictREGULATORYfindings to requirements explicitly identified by the note/caller and label them non-jurisdictional.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts` at line 114, Add jurisdiction and care-setting inputs to the flow’s advance_schema and make them available to the regulatory-analysis prompt before emitting REGULATORY findings. Alternatively, constrain REGULATORY results to requirements explicitly identified by the clinical note or caller and label those findings non-jurisdictional; preserve the existing clinicalNote validation.
114-114:⚠️ Potential issue | 🟠 MajorMission-critical: enforce the PHI boundary before Gemini execution.
advance_schemavalidates only string length and whitespace, so raw PHI can still be submitted despite the user prompt’s de-identification contract. Require an enforced pre-redaction/attestation gate or a verified compliant provider configuration before forwarding the note.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts` at line 114, Update the clinical note execution flow around the advance_schema configuration to enforce the PHI boundary before any Gemini request: require a completed pre-redaction/attestation gate or verify a compliant provider configuration, and reject or stop processing when neither condition is satisfied. Do not rely on the schema’s length or whitespace validation as de-identification.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Line 79: Update the example JSON result so its summary matches the emitted
flags: either add a corresponding CONSENT entry to the flags array, or remove
the informed-consent claim from the summary while preserving the existing
five-flag output.
---
Outside diff comments:
In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts`:
- Around line 161-181: Validate and normalize the LLM report before the
responseNode named “API Response” returns it: parse the generatedResponse from
LLMNode_453, enforce the documented schema and flagCount/ID/category invariants,
and reject malformed or markdown-wrapped output. Update the variablesNode_197
mapping so outputMapping uses only the validated report, preserving the existing
finalText/result contract for valid responses.
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Line 20: Update the DRUG_INTERACTION description in the README to describe
only medication interactions explicitly documented or suggested by the clinical
note, and clearly state that the scanner does not provide definitive or
exhaustive interaction checking.
- Around line 75-80: Update the README prose describing the flow’s result field
to state that result is a JSON string containing the serialized report, not a
JSON object. Keep the existing example and JSON structure unchanged.
---
Duplicate comments:
In `@kits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts`:
- Line 114: Add jurisdiction and care-setting inputs to the flow’s
advance_schema and make them available to the regulatory-analysis prompt before
emitting REGULATORY findings. Alternatively, constrain REGULATORY results to
requirements explicitly identified by the clinical note or caller and label
those findings non-jurisdictional; preserve the existing clinicalNote
validation.
- Line 114: Update the clinical note execution flow around the advance_schema
configuration to enforce the PHI boundary before any Gemini request: require a
completed pre-redaction/attestation gate or verify a compliant provider
configuration, and reject or stop processing when neither condition is
satisfied. Do not rely on the schema’s length or whitespace validation as
de-identification.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e59d79d3-fd4b-4c08-9b2e-259d519d411f
📒 Files selected for processing (5)
kits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.tskits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md
- Narrow DRUG_INTERACTION description in README - Clarify README result payload description as JSON string - Fix example output summary to match flags array - Add summary invariant to system prompt - Document JSON Output Validation, Jurisdiction, and PHI Enforcement limitations in agent.md and README
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
kits/clinical-note-red-flag-scanner/agent.md (1)
92-93: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy liftShared root cause: zero retention is presented as equivalent to PHI safety.
Zero retention alone does not establish provider authorization or the contractual and security controls required for PHI processing.
kits/clinical-note-red-flag-scanner/agent.md#L92-L93: require independently verified PHI-processing authorization and controls.kits/clinical-note-red-flag-scanner/README.md#L61: apply the same prerequisite wording to the setup instructions.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/agent.md` around lines 92 - 93, Update the Data Privacy guidance in kits/clinical-note-red-flag-scanner/agent.md at lines 92-93 to require independently verified provider authorization and contractual/security controls for PHI processing; do not present zero retention as sufficient. Apply the same prerequisite wording to the setup instructions in kits/clinical-note-red-flag-scanner/README.md at line 61.kits/clinical-note-red-flag-scanner/README.md (1)
31-33: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winMission brief: document the actual input schema.
The README shows
advance_schemaas{ "clinicalNote": "string" }, but the flow also enforcesminLength: 10,maxLength: 45000, and a non-whitespace pattern. Show the real constraints or clearly label the simplified shape as illustrative.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/README.md` around lines 31 - 33, Update the input schema documentation near “Flow Components” to reflect the enforced clinicalNote constraints: minimum 10 characters, maximum 45,000 characters, and at least one non-whitespace character. Alternatively, clearly label the existing shape as illustrative rather than complete.kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md (2)
37-37: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick winMission brief: remove the exact-text evidence instruction.
“Cite the exact text” conflicts with the later redaction requirement and can cause verbatim PHI or sensitive clinical text to be returned. Require only redacted/paraphrased evidence, never exact patient text or identifiers.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md` at line 37, Update the evidence guidance in the clinical note red-flag scanner prompt to remove the requirement to cite exact text. Require only redacted or paraphrased evidence in the detail and location fields, explicitly prohibiting verbatim patient text and identifiers such as names, MRNs, and DOBs.
13-13: 🔒 Security & Privacy | 🟠 Major | 🏗️ Heavy liftShared root cause: jurisdiction context is unavailable to the flow.
A disclaimer cannot make jurisdiction-specific regulatory screening reliable when the API accepts only
clinicalNote.
kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md#L13: add jurisdiction/care-setting inputs or constrain findings to note-explicit, non-jurisdictional observations.kits/clinical-note-red-flag-scanner/agent.md#L105: remove the claim that callers can supply context unless the flow accepts it.kits/clinical-note-red-flag-scanner/README.md#L86: document the same enforceable limitation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md` at line 13, Constrain the REGULATORY guidance in kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md at line 13 to note-explicit, non-jurisdictional observations, or add the required jurisdiction and care-setting inputs to the flow. Update kits/clinical-note-red-flag-scanner/agent.md at line 105 to remove the unsupported claim that callers can supply context, and update kits/clinical-note-red-flag-scanner/README.md at line 86 to document the same enforceable limitation.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Line 79: Update the example JSON in the README so the summary’s risk claims
match the listed flag categories: either remove “regulatory” from the summary or
add a corresponding REGULATORY flag, while preserving the existing category
invariant and JSON structure.
---
Outside diff comments:
In `@kits/clinical-note-red-flag-scanner/agent.md`:
- Around line 92-93: Update the Data Privacy guidance in
kits/clinical-note-red-flag-scanner/agent.md at lines 92-93 to require
independently verified provider authorization and contractual/security controls
for PHI processing; do not present zero retention as sufficient. Apply the same
prerequisite wording to the setup instructions in
kits/clinical-note-red-flag-scanner/README.md at line 61.
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md`:
- Line 37: Update the evidence guidance in the clinical note red-flag scanner
prompt to remove the requirement to cite exact text. Require only redacted or
paraphrased evidence in the detail and location fields, explicitly prohibiting
verbatim patient text and identifiers such as names, MRNs, and DOBs.
- Line 13: Constrain the REGULATORY guidance in
kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md
at line 13 to note-explicit, non-jurisdictional observations, or add the
required jurisdiction and care-setting inputs to the flow. Update
kits/clinical-note-red-flag-scanner/agent.md at line 105 to remove the
unsupported claim that callers can supply context, and update
kits/clinical-note-red-flag-scanner/README.md at line 86 to document the same
enforceable limitation.
In `@kits/clinical-note-red-flag-scanner/README.md`:
- Around line 31-33: Update the input schema documentation near “Flow
Components” to reflect the enforced clinicalNote constraints: minimum 10
characters, maximum 45,000 characters, and at least one non-whitespace
character. Alternatively, clearly label the existing shape as illustrative
rather than complete.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 5171a9b6-34bd-4598-8aa4-d5ee761ed7ed
📒 Files selected for processing (3)
kits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md
|
/validate |
|
📡 Running Studio validation — results will appear here shortly. |
|
@Vilsee LGTM! There are two coderabbit comments left, please solve them too. |
- Remove 'regulatory' mismatch from README example summary - Require verified provider authorization for PHI handling instead of assuming zero-retention is sufficient - Document full advance_schema constraints (minLength, maxLength, pattern) in agent.md and README - Prohibit verbatim citations and require paraphrased evidence in system prompt - Constrain REGULATORY findings to note-explicit non-jurisdictional observations and document schema limits
|
@akshatvirmani Thanks for the LGTM! Pushed a fix for the remaining CodeRabbit items just now (commit 28678d3) — ready for another look whenever you get a chance. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/clinical-note-red-flag-scanner/agent.md`:
- Line 92: The limitations guidance in
kits/clinical-note-red-flag-scanner/agent.md lines 92-92 must require
independently verified provider PHI authorization together with contractual and
security controls, without presenting zero retention as an alternative. Apply
the same authorization requirement to the user-facing setup guidance in
kits/clinical-note-red-flag-scanner/README.md lines 61-61.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 1adc3778-4291-4df9-8d2d-016dcf9a7877
📒 Files selected for processing (3)
kits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md
- Rewrite Data Privacy guidance in agent.md and README.md to plainly require authorization and disclaim compliance verification - Remove all references to zero-retention as an alternative pathway
|
/validate |
|
📡 Running Studio validation — results will appear here shortly. |
|
@Vilsee great! But there are 2 coderabbit comments left please fix them and then I will merge! |
- The example flags array contains no REGULATORY flag, so the summary string was updated to match the existing output consistency rule
|
@akshatvirmani Fixed the last item — README example summary is now consistent with the actual flags shown. All CodeRabbit threads resolved and ready to merge whenever you get a chance! |
|
@coderabbitai review |
✅ Action performedReview finished.
|
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md (1)
31-35: 🎯 Functional Correctness | 🟠 MajorKeep regulatory claims strictly note-explicit and non-jurisdictional.
The severity examples and README category wording still permit unsupported clinical or jurisdiction-specific conclusions. Use documentation-risk language in the prompt and describe
REGULATORYas general, note-explicit documentation expectations in the README.
kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md#L31-L35: replace “clear regulatory violation” and contraindication-based severity examples with documentation-risk examples.kits/clinical-note-red-flag-scanner/README.md#L28-L28: remove “specific regulatory requirements” and state the non-jurisdictional scope.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md` around lines 31 - 35, Update the Severity Definitions in kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md:31-35 to remove jurisdictional regulatory and contraindication-based claims, using note-explicit documentation-risk examples instead. Also update kits/clinical-note-red-flag-scanner/README.md:28 to replace “specific regulatory requirements” with a general, non-jurisdictional description of REGULATORY documentation expectations.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@kits/clinical-note-red-flag-scanner/agent.md`:
- Line 93: Update the Data Handling statement in the agent instructions to
describe only PII protections enforced by the template, removing any guarantee
about Gemini or other provider logging, storage, or repetition; explicitly
require callers to verify the provider’s retention and PHI-processing controls.
---
Outside diff comments:
In
`@kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md`:
- Around line 31-35: Update the Severity Definitions in
kits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md:31-35
to remove jurisdictional regulatory and contraindication-based claims, using
note-explicit documentation-risk examples instead. Also update
kits/clinical-note-red-flag-scanner/README.md:28 to replace “specific regulatory
requirements” with a general, non-jurisdictional description of REGULATORY
documentation expectations.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository UI (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 1e4073b4-674b-49a9-82c9-83677f563b4a
📒 Files selected for processing (3)
kits/clinical-note-red-flag-scanner/README.mdkits/clinical-note-red-flag-scanner/agent.mdkits/clinical-note-red-flag-scanner/prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md
| The constitution (`constitutions/default.md`) enforces: | ||
| - **Safety**: No harmful, illegal, or discriminatory content; refusal of jailbreaking/prompt injection attempts; uncertainty disclosure over fabrication. | ||
| - **Data Privacy**: The caller is solely responsible for ensuring the input `clinicalNote` is properly de-identified before submission, and for independently verifying that any provider processing this data has the required authorization plus contractual and security controls for handling PHI. This template performs no PHI detection, redaction, or compliance verification of its own. | ||
| - **Data Handling**: PII is never logged, stored, or repeated unless explicitly instructed by the flow; all user inputs treated as potentially adversarial. |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
Do not promise provider-side PII retention behavior.
The template sends notes to Gemini but cannot guarantee that an external provider will never log, store, or repeat PII. Scope this statement to behavior enforced by the template, and require callers to verify provider retention and PHI-processing controls.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@kits/clinical-note-red-flag-scanner/agent.md` at line 93, Update the Data
Handling statement in the agent instructions to describe only PII protections
enforced by the template, removing any guarantee about Gemini or other provider
logging, storage, or repetition; explicitly require callers to verify the
provider’s retention and PHI-processing controls.
Summary
Adds a Clinical Note Red-Flag Scanner template — a single-flow API pipeline that accepts a clinical note as text and returns a structured, severity-ranked JSON list of documentation red flags for clinician and compliance reviewer triage.
Problem Statement
Clinical documentation frequently ships with silent gaps — missing informed consent language, undocumented drug-interaction risk, incomplete vitals/history, or ambiguous dosing instructions — that create real patient-safety and regulatory exposure (aligned with EU AI Act Article 12 documentation/audit requirements, HIPAA documentation-integrity expectations, and CMS/Joint Commission accreditation standards).
The existing AgentKit registry has a
medical-assistantsymptom-checker chatbot, but nothing that performs structured compliance analysis of clinical documentation. This template fills that gap — it's not a conversational agent, it's a single-pass analysis tool that outputs machine-readable results for dashboards, alerting systems, and CDI (Clinical Documentation Improvement) workflows.What This Template Does
CONSENT,DRUG_INTERACTION,ALLERGY,VITALS,HISTORY,DOSING,FOLLOW_UP,ASSESSMENT,IDENTITY,REGULATORYCRITICAL/HIGH/MEDIUM/LOW) with per-flag reasoning, location references, and remediation recommendationsgemini-3.1-flash-lite-preview) for analysisFlow Architecture
4 nodes, single flow, template type — no UI, no env vars, no external dependencies beyond a Gemini API key.
Tested Example
Input — a 67M chest pain presentation with intentional documentation gaps:
{ "clinicalNote": "Patient: John D., 67M. Chief Complaint: Chest pain x 2 hours. History: HTN, DM2. Current Meds: Metformin 500mg BID, Lisinopril 10mg daily. Exam: BP 158/94, HR 88, RR 18. Lungs clear. Heart: regular rhythm, no murmurs. Assessment: Probable angina. Plan: Start aspirin 81mg daily, order stress test. Prescribed nitroglycerin PRN." }Output — 5 flags across 5 categories, severity-ranked:
Each flag includes
detail,location, andrecommendationfields. Full JSON output is in the README.Files Added
How It Was Built
.tscontains authentic runtime IDs and edge metadatakits/following thearticle-summarisertemplate structure exactlylamatic.config.ts,README.md,agent.md, and flow meta with comprehensive documentationChecklist
kits/article-summariser/exactlylamatic.config.ts— typetemplate, author, tags, deploy/github links.tsexported from Lamatic Studio (authentic runtime metadata)agent.md— overview, purpose, flow details (4 nodes), I/O schemas, guardrailsREADME.md— deploy button, category table, real tested example with full JSON outputconstitutions/default.md— safety guardrails and PII handlingregistry.jsonentries (verified all 79)feat: Add ...conventionclinical-note-red-flag-scanneris unique in the registrykits/clinical-note-red-flag-scanner/kit:.gitignore: ignores.lamatic/,node_modules/,.env,.env.localREADME.md: template overview + deployment/use instructions; lists 10 red-flag categories and the structured severity-ranked JSON output fields; includes de-identification/PHI-handling authorization guidance and limitations (JSON-stringresult, no compliance/jurisdiction verification, no PHI de-id enforcement)agent.md: single-pass, API-invoked AgentKit pipeline description (GraphQL trigger → Gemini analysis → variable mapping → GraphQL response); describes JSON report returned as a JSON-serialized stringconstitutions/default.md: baseline guardrails (identity/safety/data handling/tone), including refusal for prompt injection/jailbreak, no fabrication, and privacy expectations (avoid logging/storing/repeating PII unless instructed)lamatic.config.ts: Lamatic template config (metadata + one step wired toCLINICAL_NOTE_RED_FLAG_SCANNER_FLOW_ID)flows/clinical-note-red-flag-scanner.ts: flow definition exported as{ meta, inputs, references, nodes, edges }model-configs/clinical-note-red-flag-scanner_llmnode-453_generative-model-name.ts: Gemini model binding for the LLM node (gemini/gemini-3.1-flash-lite-preview)prompts/clinical-note-red-flag-scanner_llmnode-453_system_0.md: system prompt enforcing “raw JSON only” and a fixed output schema (summary,flagCount,flags[]with required fields, unique/sequential IDs, enum constraints, severity sorting, empty-state behavior, and documentation-only remediation)prompts/clinical-note-red-flag-scanner_llmnode-453_user_1.md: user prompt input contract requiring de-identified notes, embeds{{triggerNode_1.output.clinicalNote}}in a<clinical_note>block, and instructs the model to ignore instructions inside the note; reiterates no PHI detection/redactionkits/clinical-note-red-flag-scanner/flows/clinical-note-red-flag-scanner.ts)—node types + how it works:triggerNode_1(triggerNode): GraphQL/API entrypoint; requiresclinicalNote: stringwithminLength: 10,maxLength: 45000, and non-whitespace viapattern: ^.*\\S.*$LLMNode_453(dynamicNodewithnodeId: "LLMNode"): runs Gemini using the referenced system/user prompt assets and the configured generative model; producesoutput.generatedResponsevariablesNode_197(dynamicNodewithnodeId: "variablesNode"): mapsLLMNode_453.output.generatedResponse→finalTextresponseNode_triggerNode_1(responseNodewithnodeId: "graphqlResponseNode"): returnsresultas{{variablesNode_197.output.finalText}}(the JSON report delivered as a JSON string)triggerNode_1 → LLMNode_453 → variablesNode_197 → responseNode_triggerNode_1(single request/response, outputs severity-ranked red flags across 10 categories)